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ANALYSIS OF A CLASSICAL MATRIX PRECONDITIONING ALGORITHM 


LEONARD J. SCHULMAN AND ALISTAIR SINCLAIR 


Abstract. We study a classical iterative algorithm for balancing matrices in the L^o norm via a scaling 
transformation. This algorithm, which goes back to Osborne and Parlett & Reinsch in the 1960s, is implemented 
as a standard preconditioner in many numerical linear algebra packages. Surprisingly, despite its widespread 
use over several decades, no bounds were known on its rate of convergence. In this paper we prove that, 
for any irreducible n x n (real or complex) input matrix A, a natural variant of the algorithm converges in 
0{n^ log(np/e)) elementary balancing operations, where p measures the initial imbalance of A and e is the 
target imbalance of the output matrix. (The imbalance of A is maxi | log(a°“‘/a'")|, where a““‘, af are the 
maximum entries in magnitude in the ith row and column respectively.) This bound is tight up to the log n 
factor. A balancing operation scales the ith row and column so that their maximum entries are equal, and 
requires 0(m/n) arithmetic operations on average, where m is the number of non-zero elements in A. Thus 
the running time of the iterative algorithm is 0{n^m). This is the first time bound of any kind on any variant of 
the Osbome-Parlett-Reinsch algorithm. We also prove a conjecture of Chen that characterizes those matrices 
for which the limit of the balancing process is independent of the order in which balancing operations are 
performed. 


1. Introduction 

1.1. Background and discussion of results. In numerical linear algebra, it is standard practice to precon¬ 
dition an n X n real or complex matrix A by performing a similarity transform D~^AD for some diagonal 
scaling matrix D, prior to performing other computations on A such as computing its eigenvalues. The goal 
is that D~^AD should be balanced, in the sense that the norm of the ith row is equal to the norm of the ith 
column, for all i. The point here is that standard linear algebra algorithms tend to be numerically unstable 
for unbalanced matrices. Diagonal scaling achieves balance without affecting the eigenvalues of A. (Pre¬ 
conditioning typically also involves a separate process of row and column permutations which we ignore 
here.) 

This idea goes back to Osborne in 1960 ifT^ . who suggested an iterative algorithm for finding a D that 
balances A in the L 2 norm, and proved that it converges in the limit; he also proposed using the analogous 
iteration in the Lqo norm. Parlett and Reinsch generalized the algorithm to other norms iflSll . Neither 
of these papers gave any bound on the convergence time of the algorithm (in any norm). In the decades 
since then, this algorithm has been implemented as standard in almost all numerical linear algebra software, 
including EISPACK, LAPACK and MATLAB. For further background see, e.g., ll20l W2\ . Surprisingly, 
despite its widespread use in practice, no bounds are known on the running time of any variant of this 
iterative balancing algorithm. 

Our goal in this paper is to initiate a quantitative study of the Osbome-Parlett-Reinsch algorithm. We 
emphasize that our motivation is to understand an existing method which has emerged as the leading choice 
of practitioners, rather than to devise a new competitor in the asymptotic regime; however, the bounds 
we obtain show that even asymptotically this method is not far off the theoretically best (and much more 
complex) algorithms. (See the Related Work section for a discussion.) 
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most of this work was performed. AS was supported in part hy NSF grants 1016896 and 1420934. Addresses: Caltech, Engineering 
and Applied Science MC305-16, Pasadena CA 91125, USA, schulmangcaltech . edu; Computer Science Division, Soda Hall, 
University of California, Berkeley CA 94720-1776, USA, sinclairScs . berkeley. edu. 
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The iterative algorithm is very easy to describe, and involves repeated execution of a simple local balancing 
operation. Let || • || be a norm. For an index z G [n], let ||aj. || and ||a.j || denote the norms of the ith row and 
zth column of A respectively. To avoid technicalities, we make the standard assumption that A is irreducible 
(see below), which ensures these norms are always nonzero; otherwise, earlier (and faster) stages of the 
preconditioning decompose A into irreducible components. A balancing operation at i scales each entry in 
column z of A by Tj = Y/||aj.||/||a.j||, and each entry in row i by r~^ (thus leaving an unchanged). Note that 
this operation corresponds to the diagonal transformation D~^AD, where D = diag(l,..., rj,..., 1). The 
iterative algorithm simply performs the following step repeatedly {^P/ck an index i and perform a balancing 
operation at i. 

We focus in this paper on the case of balancing in the Loo norm, where the goal is to find D so that in 
D~^AD the largest entry (in magnitude) in the zth row is equal to the largest entry in the fth column, for all i. 

We refer to such a matrix as balanced. The scaling factor in the algorithm is then just r* = where 

= maxj | Oij \ and a“ = maxj | aji \ are the maximum elements in the ith row and column respectively. 
This version of the algorithm is particularly simple to implement, which makes it an attractive alternative 
to (say) the L 2 version. Since the algorithm depends only on the magnitudes of the aij, and multiplies 
them only by positive reals, we will simplify from now on by assuming that all entries Oij are non-negative, 
keeping in mind that the balancing operations are actually performed on the original matrix 

It is instructive to view this procedure in terms of the directed weighted graph Ga, in which there is an edge 
{i, j) of weight aij if Oij > 0 (and no edge if Oij = 0). Irreducibility of A corresponds to Ga being strongly 
connected. The above balancing operation scales all incoming (respectively, outgoing) edge weights at 

vertex i by rj = (respectively, r“^), where 0 “ are respectively the maximum outgoing 

and incoming weights at i. At first sight (taking logarithms of the edge weights) this process resembles a 
diffusion on Ga', however, the fact that the scaling at each step depends on the maximum incoming/outgoing 
edge weight is a nonlinearity that makes the process much harder to analyze. 

Clearly any fixed point of the above iteration must be a balanced matrix A, i.e., = a™ for all r, in other 

words, in the weighted graph Ga, the maximum incoming edge weight is equal to the maximum outgoing 
edge weight at every vertex. Note that the product of the edge weights along any cycle in Ga is an invariant 
of the algorithm. It is thus natural to call two matrices equivalent if they agree on all these invariants. 
The fact that the algorithm always converges asymptotically to a balanced matrix was proved, surprisingly 
recently, by Chen ll2]0 in particular this implies the existence of a balanced matrix in each equivalence 
class. Chen also conjectured that a worst-case input (in terms of rate of convergence) is one in which Ga 
is simply a directed cycle; in this case each vertex has only one incoming and one outgoing edge, so the 
balancing operation is linear and convergence is easily seen to occur within 0(n^) operations. (The 0 here 
hides factors that depend on the maximum imbalance in the input matrix and the desired bound on how close 
the output matrix is to being balanced.) In most cases the process seems to converge much faster, but the 
problem of analyzing the convergence rate in general graphs, or even in any strongly connected graph other 
than a directed cycle, has remained open until now. For example, even the case of two directed cycles that 
share a common vertex is quite non-trivial and has proved resistant to standard arguments. 

The first difficulty one faces in analyzing convergence rates is that the balanced matrix to which the algorithm 
converges is in general not uniquely defined, but may depend on the sequence of indices i selected. (See 
Figure [T] for a simple example that illustrates this phenomenon.) Our first result proves a conjecture of 
Chen lO, which characterizes all cases in which the balancing problem has a unique solution. For a balanced 


^The algorithm in I14II15I operates on the indices in a fixed cyclic order. For ease of analysis we pick indices randomly. We 
conjecture that this change makes little difference to the rate of convergence. 

^It is sometimes also assumed that the diagonal entries of A are replaced by zeros before the balancing process, since balancing 
never alters them. This replacement may of course change the problem since (e.g.) a diagonally dominant matrix is already 
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Figure 1. Example of a non-UB matrix A. The balanced matrix Bi results from A by 
balancing at indices 1,4, while B 2 results by balancing at indices 2,4. 


matrix B = {bij} and any real w > 0, let denote the subgraph of Gb consisting of those edges of 
magnitude at least w, with isolated vertices removed. (A vertex with a self-loop is not considered isolated.) 

Theorem 1. A balanced matrix B is the unique balanced matrix in its equivalence class if and only if 
is strongly connected for all w. 

Note that this result gives an implicit criterion for whether a given input matrix A can be uniquely balanced: 
namely, that it is equivalent to a balanced matrix B with the above property. We say that such matrices A 
satisfy the Unique Balance (UB) condition. 

In an earlier version of the present paper ifT/l . we focused on the UB case and showed that a certain variant 
of the Osbome-Parlett-Reinsch algorithm converges in 0{n^ logn) balancing operations on any UB input 
matrix A. In this paper we extend that analysis to all input matrices, for a slightly different variant of the 
algorithm. (The term “variant” here refers only to the order in which balancing operations are performed.) 

To describe this variant, we need the notion of “raising” and “lowering” operations, which are one-sided 
versions of the standard balancing operation. A raising operation at i performs a standard balancing oper¬ 
ation at i if > < and does nothing otherwise; similarly, a lowering operation at i performs a standard 
balancing operation at i if < a“ and does nothing otherwise. Our variant of the algorithm will perform a 
sequence of random raising operations followed by a sequence of random lowering operations; equivalently, 
it can be viewed as the standard algorithm (with a random index order) in which some of the balancing 
operations are censored according to a very simple condition. 

To specify the algorithm precisely, we also need a measure of how far a matrix is from balanced. We 
thus define the imbalance of A as max* | log(a™Y®r)l’ ^ e-balanced if its imbalance is at 

most £. The target balance parameter e > 0 is provided to the algorithm as an additional input. We also 
need one-sided versions of these definitions: namely, A is e-raising-balanced if max* log(aYV“r) — ^ 
raising-balanced if e = 0), and e-lowering-balanced if maxj log(a“/aY*) < e (or lowering-balanced if 
e = 0). Plainly A is e-balanced iff it is both e-raising-balanced and e-lowering-balanced. 

We note that even the asymptotic convergence of this variant of the algorithm is not immediately clear, and 
does not follow from Chen’s proof O or from our proof in Appendix The reason is that those proofs 
assume that the sequence of balancing operations is fair, in the sense that each index i appears arbitrarily 
often. In the two-phase version, fairness cannot be guaranteed since some balancing operations are censored. 
Instead, convergence of the two-phase algorithm is an immediate consequence of the following theorem. 

Theorem 2. For any input matrix A, any fair sequence of raising (resp., lowering) operations converges to 
a unique raising-balanced (resp., lowering-balanced) matrix, independent of the order of the operations. 


balanced in Loa- We do not assume such a replacement has been made; our analysis of the balancing algorithm applies whether or 
not diagonal entries are zero. 

^To make the current paper self-contained, we present in Appendix|^an alternative proof of convergence that uses combinatorial 
rather than topological techniques and is closer to the methodology developed in this paper. 
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Algorithm 1: 2PhaseBalance(A, e) 

Input: An n x n matrix A with (unknown) imbalance p; and a target e > 0. 
Output: A matrix equivalent to A that is e-balanced with high probability. 

1 Compute p in time 0{n^) 

2 Repeat T = 0(n^ log(p/e)) times //Raising Phase// 

3 Pick an index i u.a.r. and apply a raising operation at i 

4 Repeat T = 0(n^ log(p/e)) times //Lowering Phase// 

5 Pick an index i u.a.r. and apply a lowering operation at i 

6 Output the resulting matrix 


Correctness of the two-phase algorithm follows because it is easy to see that the lowering phase cannot 
increase the raising imbalance of A (see Proposition |^. The value T is chosen to guarantee that A is e- 
raising-balanced (resp., e-lowering-balanced) at the end of a phase w.h.p.; hence, at the conclusion of both 
phases A will indeed be e-balanced w.h.p. 

Theorem]^ is actually a somewhat surprising result: even when A is not UB (so that the balanced matrix to 
which the original version of the algorithm converges depends on the sequence of operations performed), if 
we perform only raising (or only lowering) operations we are guaranteed a unique limit. It is this fact that 
will allow us to obtain an analysis of the rate of convergence. 

We are now able to state the main result of the paper, which bounds the rate of convergence of the above 
algorithm. 

Theorem 3. On input {A, e), where A is an arbitrary irreducible non-negative matrix with imbalance p, the 
above two-phase L^o balancing algorithm performs 0{n^ log{np/e)) balancing operations and outputs a 
matrix equivalent to A that is e-balanced w./i.p|^ 

This is the first time bound (except on the cycle) for any variant of the Osborne-Parlett-Reinsch algorithm 
(under any norm). Moreover, the bound on the convergence rate is actually tight up to a factor 0(log n), in 
view of the Q(n^) lower bound we mentioned earlier for the cycle. Indeed, Theorem|^implies that the cycle 
is a worst case input for the algorithm (up to the log n factor), as conjectured by Chen ||2l. 

Remark. Throughout the paper, we quantify the rate of convergence of the algorithm in terms of the number 
of balancing operations performed. Since each balancing operation (raising or lowering) involves finding 
a maximum elemenf in one row and column and fhen scaling fhe row and column, if can be performed 
on average in 0{m/n) arifhmefic operafions, where m is fhe number of edges in Ga (i-e., the number of 
non-zero entries in A). Thus, in terms of arithmetic operations, the (average) running time of the algorithm 
is 0(n^mlogn). 

We close this section with a brief outline of our approach to analyzing the algorithm. As we have observed 
above, it suffices fo analyze only fhe raising phase; by symmefry, an idenfical analysis applies fo fhe lowering 
phas^ The firsl key observation is fhaf fhe uniqueness of fhe limif in fhe raising phase allows us fo assign 
a well-defined height fo every verfex in Ga', the heighf of a verfex measures fhe “amounf of raising” fhaf 
needs fo be performed af fhaf verfex in order fo reach fhe unique raising-balanced configurafion. Each raising 
operafion can fhen be viewed as smoofhing fhese heighfs locally. In a diffusion process fhis nafurally leads fo 
an associafed Laplacian pofenfial funclion consisfing of fhe sum of squares of local heighf differences, whose 
convergence is capfured by fhe eigenvalues of fhe heal kernel operafor. Affempfs fo conducl a similar analysis 
here fail. One serious problem is fhaf a balancing operafion al verfex i depends only on fhe incoming and 

“^We use the phrase “with high probability (w.h.p.)” to mean with probability tending to 1 as n —>■ oo. 

^It is an interesting open question whether the introduction of phases in the Osborne-Parlett-Reinsch algorithm leads to more 
rapid convergence in practice. 
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outgoing edges of maximum weight, but has side effects on all other edges incident at i. Another problem 
is that there are, essentially, “levels” within the graph: roughly speaking, vertices that lie in the “lower” 
levels of the graph cannot reliably converge toward (raising) balance until the vertices in the “higher” levels 
have converged. While phenomena similar to this can arise in diffusion processes, in our nonlinear setting 
standard tools such as eigenvalues are lacking to capture them. 

The chief technical challenge in our analysis is to relate the local height changes achieved by raising oper¬ 
ations to an improvement in a global potential function (the sum of the heights), thus ensuring significant 
progress (in expectation) over time. The details of our analysis ensure an expected improvement that is a 
factor n(n^) of the current value of the potential function in each step, thus leading to global convergence 
in O(n^) raising operations. 

To establish the above local-global connection, we need a two-dimensional representation of the current 
state of the algorithm, which records not only the height of a vertex but also its level; unlike the heights, 
which change over time, the level of a vertex is fixed. This two-dimensional representation leads naturally 
to the notion of the momentum of a vertex, which measures how far a vertex currently is from its true level. 
Momentum is a key ingredient in relating local improvements to the global potential function. 

1.2. Related work. As mentioned above, the idea of iterative diagonal balancing was introduced by Os¬ 
borne lIT^ in 1960, who was motivated by the observation that minimizing the Frobenius norm of A is 
equivalent to balancing A in the L 2 norm. Osborne formulated an L 2 version of the above iterative algo¬ 
rithm and proved that it converges in the limit (but with no rate bound); he also proposed the Loo algorithm 
discussed in this paper but left the question of convergence open. Convergence was first proved by Chen |2| 
almost 40 years later. Parlett and Reinsch ifTSl generalized Osborne’s algorithm to other norms (without 
proving convergence) and discussed a number of practical implementation issues for preconditioning. For 
the Li version of the Osborne-Parlett-Reinsch algorithm, convergence was proved by Grad [161, uniqueness 
of the balanced matrix by Hartfiel Q, and a characterization of it by Eaves et al. ||4l, but again no bounds 
on the running time of Li balancing were given. 

Despite the scarcity of theoretical support, these algorithms are implemented as standard in many numerical 
linear algebra packages and experience shows them to be both useful and fast. 

A sequence of two papers offers an alternative, and substantially more complex, non-iterative algorithm for 
matrix balancing in the Lqo norm. Schneider and Schneider ifT^ gave an O(n^) -time algorithm based on 
finding maximum mean-weight cycles in a graph; the running time was improved to 0{nm -|- logn) 
using Fibonnaci heaps and other techniques by Young, Tarjan and Orlin 11211 . This is asymptotically faster 
than the O(n^m) worst case running time we establish for the iterative algorithm; and for some graphs (e.g., 
the cycle) it is faster than the actual running time of the iterative algorithm. 

However the iterative method has been favored in practice. This may be justified by the empirical distribution 
of inputs. Moreover, it is definitely driven by the fact that the iterative method offers steady partial progress, 
and so can deliver, without being run to completion, a matrix that is sufficiently balanced for the subsequent 
linear algebra computation. In practice indeed the method is usually run for far fewer iterations than are 
needed in the worst case. Our purpose in this paper is to provide the first theoretical understanding of the 
widely used iterative methods, rather than to derive new theoretical bounds for the underlying problem. 

In other work on matrix balancing, Kalantari et al. ifTOl considered balancing in the Li norm, and provided 
the first polynomial time algorithm by reduction to convex programming: 0{rA) via the ellipsoid method. 

Diagonal scaling has also been used to minimize matrix norms without regard to balancing. For example, 
Strom IIT 9 II considers the problem of finding diagonal scaling mafrices to minimize the max and Frobenius 
norms of the matrix. In particular, for the max norm he proves that (when A is irreducible) the optimal di¬ 
agonal scaling matrix is obtained from the principal eigenvector. Chen and Demmel Q show that a suitable 
notion of weighted balancing can be used to minimize the 2-norm, and discuss Krylov-based algorithms 
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that work efficiently on sparse matrices. Boyd et al. ||T1 formulate the problem of minimizing the Frobenius 
norm as a generalized eigenvalue problem. 

A different notion of matrix balancing is often known as Sinkhorn balancing after Sinkhorn lITSl . who pro¬ 
posed a natural iterative algorithm analogous to that of Osbome-Parlett-Reinsch. The goal here is to find 
a scaling matrix D such that D~^AD has prescribed row and column sums. A polynomial time algorithm 
for this problem was given by Kalantari and Khachian f9i], while the convergence rate of Sinkhorn’s it¬ 
erative algorithm was studied by several authors l|5l [H |T3l [TTl. In particular Linial, Samorodnitsky and 
Wigderson ifT^ gave strongly polynomial bounds (after a non-Sinkhom preprocessing step) and derived a 
surprising approximation scheme for the permanent of a non-negative matrix. 

2. Preliminaries 

In this section we introduce some terminology and notation that we will use throughout the paper. We begin 
with an equivalent reformulation of the Osbome-Parlett-Reinsch balancing algorithm from the Introduction. 

Let A be an irreducible n x n matrix. As described in the Introduction, the algorithm operates by iteratively 
picking an index i and geometrically averaging the maximum entries (in magnitude) and a™. It is 
convenient to switch to arithmetic averages by setting = log \ aij\ (so that 0 entries of A become —oo). 

Letting a™' = maxj Uij and aj" = maxj Oji, a balancing operation at i then consists of adding — 
cA^)(2 to the ith column and subtracting the same quantity from the ith row. (In the actual implementation 
of the algorithm, this corresponds to the multiplicative diagonal transformation D = diag(l,..., r^,..., 1) 
applied to the original matrix A. However, for the rest of the paper we will assume that the algorithm 
operates additively on the logarithms of the matrix entries and ignore the details of this trivial translation.) 

We re-use the notation Ga from the Introduction to denote the digraph with edges (f, j) such that aij > —oo. 
(The passage from roman to greek letters will always resolve this abuse of notation.) Irreducibility of A again 
corresponds to Ga being strongly connected. We shall regard a as a real-valued function defined only on 
fhe edges of Ga, and call it a graph function. Note that if balancing operations change the graph function a 
to a' we always have Ga = Ga'- For the remainder of the paper, we use letters u, v etc. rather than i,j to 
denote vertices as our focus will be on the graph Ga rather than on the matrix A. 

We say that a graph function a is balanced if = a™ for all v. For each v, define p^ = — a“|- The 

imbalance of a is /? := max^ py, and a is e-balanced if p < s. (These definitions are equivalent to those in 
the Introduction in terms of the matrix entries Oij .) A balancing operation is available at v iff py > 0. 

As indicated in the Introduction, we analyze a slightly modified version of the basic Osborne-Parlett-Reinsch 
algorithm, which has two phases and performs only certain balancing operations (raising or lowering) during 
a phase. Let p^ = max{0, — a™} and py = max{0, a™ — be the raising and lowering imbalances, 
respectively, at v. Note that py = Py + Py = max{p^,/ 9 ^}, since one of p^, p^ must be zero. A 
raising operation at v performs a standard balancing operation at i; if > 0 and does nothing otherwise; 
similarly, a lowering operation at v performs a standard balancing operation at u if >0 and does nothing 
otherwise. Such an operation is said to raise (resp., lower) v by an amount Py /2 (resp., p^/2); note that 
a raising operation at v reduces its raising imbalance to zero unless the maximum outgoing edge from v is 
a self-loop, in which case the raising imbalance is reduced by a factor of 2. The raising imbalance of a is 
pR ._ j] 2 a.x„ p^, and the lowering imbalance of a is p^ := max^ p]f. We say that a is e-raising-balanced 
if p^ < e, and e-lowering-balanced if p^ < e. Clearly a is e-balanced iff it is both e-raising-balanced and 
e-lowering-balanced. 

For any real w, denote by Ga the subgraph of Ga consisting of edges {u, v) such that Uyv > w, with isolated 
vertices removed. (A vertex with a self-loop is not considered isolated.) 

Let a, 7 be two graph functions on the same graph G. We say that a, 7 are equivalent, a ~ 7 , if their sums 
around all cycles of G are equal, i.e., if ~ = 0 for all cycles (ui,..., Vk), where 

Vk+l = Vl- 
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It is convenient for our analysis to introduce a generalization of the standard local balancing operation at a 
vertex. Let a be a graph function. For any x G M and 5 C [n], denote by a + xS (“a shifted by x at the set 
S”) the graph function given by (a + xS)uv = otuv + x ■ {Is{u) — Is{v)), where Is is the indicator function 
for membership in S. Thus the balancing operation at v is equivalent to shifting by (a“ — at the 

set {x}. It is readily seen that two graph functions are equivalent iff there is a sequence of shifts converting 
one to the other. Note that, like balancing operations, shifts of a do not change Ga- 

3. The Unique Balance property 

As shown by Chen (and reproved with a different argument in the present paper—see Appendix]^, balanc¬ 
ing operations do not cycle; Chen argued this topologically while we show that there is an order on graph 
functions such that nontrivial balancing operations are strictly lowering in the order. However, neither the 
topological nor the order-theoretic argument yields useful quantitative information on the rate of conver¬ 
gence. One reason for this, but not the only reason, is that the limit point of the process is not in general 
uniquely determined by the input matrix A (see Figure[T]for a simple example). 

For a broad class of inputs, however, a unique limit does exist. The characterization of this class was 
conjectured by Chen O and is our first result. This was already stated as Theorem[T]in the Introduction; we 
restate it here in slightly different notation. 

Theorem A balanced graph function (3 is the unique balanced graph function in its equivalence class if 

and only ifG^ is strongly connected for all w. 

Proof of Let /3 be balanced. Suppose G'Jj’ is not strongly connected, and that w is the largest value for 
which this is the case. Note that since /? is balanced, G'^ has no sources or sinks, and hence must contain 
at least one source strongly connected component (see) S and one sink see T, both of which are non-trivial 
(i.e., contain at least two vertices or one self-loop). Moreover, by maximality of w, the weights of external 
edges (between sec’s) in G'Jj’ are all exactly w. Choose vertices s £ S and t £ T, and consider a directed 
simple path P from s to f in Gp. (Such a path exists because Gp is strongly connected.) Let e > 0 be 
smaller than the gap between any two distinct values of /3. Then in G^j^sT, the length of P is e less than it 
is in Gp. 

Now apply balancing operations in any order to the graph function (3 + eT (which is equivalent to /?), to 
balance it (in the limit). We claim that no balancing operation will ever be applied at any vertex inside a 
non-trivial see of G^, and hence at either s or t. To see this, note that such a balancing operation can only 
occur if the weight of some external edge incident on the see reaches a value larger than w. But in /3 + eT no 
external edge has weight larger than w (this was true in (3 and the shift at T has increased the weight only of 
edges leaving T, but to a value less than w), so balancing operations at the vertices outside the sec’s cannot 
increase the weight of edges above w. Hence in the limiting (and hence balanced) graph function, the length 
of the path P from s to f will remain e less than it is in (3. (The length of a path can be altered only by a 
balancing operation at one of its endpoints.) Hence this limiting graph function is different from f3. 

Proof of Let a ~ /3 be distinct balanced graph functions. We show that there is a w such that neither 
G'p nor G^ is strongly connected. 

Let w be the largest value such that G^ G^ (the supremum is achieved as the graph is finite), and 
suppose w.l.o.g. there is an edge (n, v) in G^ — G^. Then G^ cannot be strongly connected because if 
it were, consider any cycle in GJj" which includes {u, v). The weight of this cycle is larger in Gp than in 
Ga because all weights larger than w in the cycle are identical in both functions, and in G^ all remaining 
weights equal w, while in Ga all are at most w and at least one is strictly less than w. Hence a /3, which 
is a contradiction. 

Likewise, if there is an edge in G^ — G^ then G^ is not strongly connected. This leaves only the case 
Ga C GJj. Let U be the set of vertices from which u can be reached in G^, and V be the set of vertices 
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reachable from v in G^. As shown in the previous paragraph, these sets are disjoint. Each set must contain 
a non-trivial sec (because /3 is balanced and therefore contains no sources or sinks). These sec’s (call 
them U' and V’) must also be strongly connected in in fact, we argue that a equals /3 on the edges of 
within each sec. For, consider any cycle using edges of G^ in the sec. The cycle may contain edges 
heavier than w (which are identical in a and /3), and some edges that are equal to w in /3 and < w in a. But 
then since a ^ j3, these edges must also equal w in a. So a and (3 are identical on the edges of G^ in the 
sec. Now, if G^ is strongly connected, then it contains a path from V' to U'. Then since G^ C G^, this 
path also lies in G^. Contradiction. □ 


In light of Theorem [T] we say that a graph function a has the Unique Balance (UB) property if there is a 
balanced /3 ~ a such that G^ is strongly connected for all w. By the convergence theorem (see AppendixjA i, 
on any input matrix with the UB property the original Osborne-Parlett-Reinsch balancing algorithm ^1 
converge to a unique balanced matrix. In an earlier version of the present paper |[T7]| . we analyzed the rate 
of convergence of (a suitable variant of) the algorithm in the UB case, and derived the same bound as we 
now obtain for general matrices in Theorem In the remainder of this paper, we turn our attention to the 
general case and leave Theorem [T] as an interesting structural result. 


4. Convergence oe the two-phase algorithm 

The goal of this section is to prove that the two-phase variant of the iterative balancing algorithm described 
in the Introduction converges and outputs a e-balanced graph function. Note that convergence does not 
follow from the general convergence proof given in Appendix]^ since the sequence of balancing operations 
in this case is not “fair” in the sense that every vertex is guaranteed to appear arbitrarily often. 

Convergence of the two-phase variant follows immediately from Theorem[^stated in the Introduction, which 
we restate here in graph function terms. 

Theorem For any initial graph function a, any fair sequence of raising (resp., lowering) operations 
converges to a unique raising-balanced graph function (resp., lowering-balanced graph function a^), 
independent of the order of the operations. 

To deduce that the output of the two-phase algorithm is indeed e-balanced, we need the following additional 
simple observation. 

Proposition 4. A lowering operation cannot increase any raising imbalance p^. Symmetrically, a raising 
operation cannot increase any lowering imbalance p^. 

Proof. We give the proof only for lowering operations; the proof for raising operations is symmetrical. 
Consider a lowering operation at vertex v, and assume that it has a non-zero effect, i.e., > 0. Lowering 

at V reduces p^ (to zero unless the maximum incoming edge at u is a self-loop in which case the reduction 
is to Py / 2) and keeps p^ at zero. The only other imbalances that can be affected are at neighbors of v. 
Lowering at v decreases edge weights auv, which cannot increase p^. Similarly, lowering at v increases 
edge weights which again cannot increase p^. □ 

It is now easy to see that the two-phase algorithm terminates with an e-balanced graph function. By The¬ 
orem]^ each of the two phases terminates with the appropriate balance condition. At the end of the raising 
phase the function is e-raising balanced, and by Proposition]^ it remains so throughout the lowering phase. 
At the end of the lowering phase the resulting graph function is both e-raising balanced and e-lowering 
balanced, and hence e-balanced as required. 

The remainder of this section is devoted to proving Theorem]^ We focus on raising operations; the proof 
for lowering operations follows by a symmetrical argument. 
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We introduce the following notation. Fix the initial graph function a and a particular fair infinite sequence of 
raising operations. Let a = a(0), a(l),..., a{t), ... be the resulting sequence of graph functions obtained. 
Also, for each f > 0 let the raising vector r{t) = {rv{t)) record the cumulative amount by which each vertex 
has been raised. I.e., r(0) = 0 and, if the tth raising operation is at vertex v, then r„(f) = rv{t — 1) + Py/2 
and ru{t) = — 1) for all u ^ v. Note that the vector r{t) completely specified a (f) via the relation 

(1) Oiuv{t) = a«^,(0) + ry{t) - ru{t). 

Lemma 5. The sequence r{t) is increasing and bounded above, and hence converges to a limit vector r* as 
f —>■ oo. 

Proof. The fact that r{t) is increasing is immediate from its definition. To see that it is bounded, observe 
first that the graph function a{t) is bounded above and below for all t; indeed, it is immediately clear from 
the definition of balancing that max^^^ Oiuv{t) cannot increase with t, while the less obvious fact that auv{t) 
is bounded below follows from Lemma [TT] in Appendix [A| 

Now define the function Nuv{t)\auv{t), where Nuy{t) is the number of edges {u', v') 

of weight au'v'{l) < ctuvit)- We claim that, if the fth raising operation is at vertex v, the change in is 
bounded by 

(2) ch(t + 1) - m < -p^/2. 

Since by the observations in the mevious paragraph <l>(f) is always bounded, and the increase in r^(f) is 
exactly / 0 ^/ 2 , we conclude from (0 that r^{t) is also bounded. 

To prove Q, represent each edge {u,v) by a particle on the real line at position auv{t). We will view 
[f, t+1] as a continuous time interval, during which each particle (u, v) moves at constant speed from auv{t) 
to auv{t + 1). We now compute the rate of change of under a raising operation at vertex v (assuming that 
> 0 so that the raising operation is non-trivial; otherwise, does not change). The only particles that 
move are (u, v) and (u, w) as u, w range over neighbors of v, each particle (n, v) moves a distance p^/2 to 
the right, and each (u, w) moves the same distance to the left. Let u, w be such that auv = max(„ j,) auv{t) 
and avio = max(^ otvw{t), so that p^ = ay^it) — auy{t). The rate of change of d>(f') for t' in the interval 
[t, f + 1) is given by 

{u,v) 

R 

< ^{{Nyiy{t') - l){Nyiy{t') - l)! - Ny^yit'f) 

pR 

^ _ rv 

- 

The negative term in the second line arises from the contribution of the particle {v, w), whose edge remains 
heaviest throughout. The positive term arises because there are at most Nyy,{t') — 1 particles to the left of 
{v, w), each of which contributes at most {Nyiy{t') — 1)!. 

As the particles move and pass other particles, the coefficients Nuy{t') change. However, observe that the 
above calculation is valid at all times f G [f, f +1), since it relies only on the fact that (u, w) is the rightmost 
moving particle. Hence we deduce the bound Q claimed earlier, which completes the proof. □ 

It follows immediately from Lemmaand equation ([T]) that a{t) also converges to a limit a*, and by con¬ 
tinuity and fairness a* must be raising-balanced. It remains to show that a* is independent of the sequence 
of raising operations. For this we require the following monotonicity property of raising operations. 

^However, the converse is not quite true; given a{f) we can deduce r{t) only up to an additive constant. 
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Lemma 6. Let r, s be raising vectors obtained from a common initial graph function a via different se¬ 
quences of raising operations. Suppose that r < s, and let r', s' be the new raising vectors obtained from 
r, s after one additional raising operation at a common vertex v. Then r' < s'. 


Proof. Clearly we have r'^ = Vu < Su = for all u v,so we need only show that By definition 

of the raising operation at v, we have (using equation Q) 

R 

/ I rv 

r,, = r.u + — 


= + - maxjO, max - r^) - max {auv + Vy - r„)} 

2 w.v^w u\u^v 

= max{2r^, max + r^) + min (r„ - auv)}- 

} '' ILW _ ^U) 'tf'ii _ 


(Here, and subsequently, we use the notation u —)• u to indicate that there is an edge {u, v) in Ga.) Since the 
right-hand side here is monotonically increasing in all entries of r, and r < s, we conclude that r'y < s'y as 
required. □ 

Lemma 7. For any initial graph function a and sequence of raising vectors r(l), r(2),..., r’(f), ... ob¬ 
tained from a fair sequence of raising operations, the limit r* = limt_>.oo r{t) exists and depends only on a, 
not on the sequence of raising operations. 

Proof. Consider some fair sequence of raising operations starting from a, and let a{t), r{t) be the cor¬ 
responding sequences of graph functions and raising vectors respectively. By Lemma the limits a* = 
limj_^oo 0 ({t) and r* = limt_ 5 .oo r{t) exist and a* is raising-balanced. 

Now consider a second sequence of raising operations also starting from a, and let s(t) be the corresponding 
sequence of raising vectors. Again, by Lemma]^ the limit s* = limt_>.oo s(t) exists. 

Suppose we apply this second sequence of raising operations, starting on the one hand from a and on the 
other hand from a*. The resulting raising vectors (with respect to a) are respectively s* and r* (the latter 
because a* is already raising-balanced). Applying Lemma to compare the raising vectors at each step 
of this process yields s* < r*. But by symmetry we must also have r* < s*, and hence r* = s*. This 
completes the proof. □ 

Since the graph function a* is completely determined by the raising vector r*, Lemma |7j and its symmetric 
counterpart for lowering operations, proves Theorem]^ which was the main goal of this section. 

Figurej^shows an example of a graph function and its associated unique corresponding raising- and lowering- 
balanced functions. 


5. Rate of convergence 

In this section we will prove the main result of the paper. Theorem from the Introduction, which gives a 
tight bound on the rate of convergence of the iterative balancing algorithm. We restate the result here for 
convenience. 

Theorem 1^ On input {A, e), where A is an arbitrary irreducible non-negative matrix with imbalance p, the 
above two-phase balancing algorithm performs 0(n^ \og{np/e)) balancing operations and outputs a 
matrix equivalent to A that is e-balanced w.h.p. 

We give here a brief, high-level outline of our analysis. First note that, by Proposition in the previous 
section, it suffices to analyze each phase of the algorithm separately. Accordingly, we will analyze only 
the raising phase, showing that it achieves an e-raising-balanced graph function w.h.p. after 0(n^ \og{p/e)) 
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Figure 2. Example showing convergence of raising and lowering operations, (i) A graph 
function a. (ii) The raising-balanced graph function obtained in the limit from a by 
raising operations; numbers in square brackets at the vertices show the limiting raising 
vector r*. (Hi) The balanced graph function (a^)^ obtained in the limit by applying low¬ 
ering operations to a^, together with its associated lowering vector. (This is the unique 
balanced graph function to which the two-phase algorithm “converges” if both phases are 
run to convergence.) Figures (iv) and (v) show the graph functions and (a^)^ respec¬ 
tively (and their associated lowering/raising vectors), obtained by reversing the order of 
the raising and lowering phases. Note that the balanced graph functions (a^)^ (in (Hi)) 
and (a^)^ (in (v)) are not the same; while both versions (raising-lowering and lowering¬ 
raising) converge to a unique balanced graph function, these limits are not equal. The 
graph function a is not UB. 

raising operations. By symmetry the same bound holds for the lowering phase: simply observe that lowering 
on the graph Ga is equivalent to raising on the transpose graph G^. 

In the Introduction we already discussed some of the difficulties which prevent us from analyzing the dy¬ 
namics of the algorithm by tracking a scalar quantity at each vertex, as would be the case for the analogous 
but much simpler heat-kernel dynamics. A key ingredient in our solution to the problem is to introduce a 
two-dimensional representation of the graph function a, together with an associated potential function that 
will enable our analysis of the raising phase. One coordinate (the vertical coordinate) in this representation, 
called the “height” of a vertex, is based on the unique raising vector r* whose existence is guaranteed by 
Lemma 1^ of the previous section; the second (horizontal) coordinate is the “level” of the vertex, as alluded 
to in the Introduction. This representation, which is specified in defail in Section |5.1[ will be insfrumenfal 
in obfaining upper and lower bounds relafing fhe (global) pofenfial funclion fo fhe local imbalances (which 
govern fhe effecf of a single raising operafion); fhese bounds are derived in Sections [5^ and [53] respecfively. 
Finally, in Section [5^ we conclude fhe proof of Theorem 
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5.1. A two-dimensional representation. Throughout this and the next two subsections, we focus exclu¬ 
sively on the raising phase. As we observed above, exactly the same rate of convergence will apply to the 
lowering phase. 

Recall from Lemma |7] that, for any initial graph function a, the limiting raising vector r*, which records 
the asymptotic total amount of raising performed at each vertex by any infinite fair sequence of raising 
operations, is uniquely defined and specifies a unique raising balanced graph funcfion a^. We may fherefore 
use r* fo define a height function 

Vv ■■= -r*, 

i.e., —zjy is fhe amounf of raising fhaf remains fo be performed af verfex v in order fo reach fhe raising- 
balanced graph funcfion a^. (Nofe fhaf, throughouf, all heighfs are non-posifive and converge monofonically 
fo zero. This convenfion furns ouf fo be convenienf in our analysis.) The heighfs are relafed fo a and by 

(3) a^y-auv = yu-yv, 

which is jusf a restafemenf of equafion Q- 

In addifion fo fhe heighfs y^, we will need fo infroduce a second coordinafe Xy defined by 

(4) Xy := max a^y. 

u:u^v 

We refer fo Xy as fhe level of verfex v. Nofe fhaf (infuifively af least), lower level vertices can only reliably 
be close to raising-balanced after higher level ones are close, so the levels capture one of the distinguish¬ 
ing features of the problem. The two-dimensional representation {xy,yy) of vertices v will be key to the 
remainder of our argument. 

Figure shows the two-dimensional representation of the graph function in Figure [^i). The horizontal 
coordinates can be read off from the maximum incoming edges at each vertex in Figure |^ii), and the 
vertical coordinates from the square brackets in the same figure. 


/' 

>- i 

'd b' 

»- 


c 

a 


e, 









Xv —32 —4 Xv —5 


Figure 3. Two-dimensional representation of the graph function a shown in Figure^i). 

We will measure progress of fhe raising phase by means of fhe global pofenfial funcfion 4^ = —'ffyy, 
which is fhe fofal amounf of raising fhaf remains fo be done fo reach a^. In order fo relafe iF fo fhe local 
imbalances p^, if will be convenienf fo infroduce a closely relafed global quantify h, which is jusf fhe 
difference befween fhe maximum and minimum heighfs. Thaf is, setting ymin = minu pu and ymax = 
max„ pu, we define 

h •— I/max f/min' 
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(As we shall see shortly (Corollary [TT|), it is always the case that ymax = 0 so in fact h = —yminO Note that 
ih and h differ by at most a factor of n, namely: 

(5) 'h > > -'h. 

n 

The key ingredients in our analysis are the following two bounds relating h to the local imbalances p^. 
Proposition 8. For any graph Junction a, we have p^/2 < h. 

Proposition 9. For any graph function a, we have h < {n — 1) p^. 

Proposition confirms that, to ensure e-raising-balance, it suffices to achieve ip < e/2. The much more 
subtle Proposition 1^ implies that ih decreases at each step by an expected (1 — n(^)) factor. To see 
this implication, note that the expected decrease in ih in one raising operation is ^ p^, and apply 

Proposition]^ and equation Q. 

In the next two subsections we prove Propositions]^ and [^respectively. 

5.2. Proof of Proposition]^ The proposition will follow almost immediately from the following straight¬ 
forward lemma relating heights to local imbalances. Since we are dealing exclusively with the raising phase, 
in this and the next two subsections we will write p^, pin place of p^, p^ to simplify notation. 

Lemma 10. Let a be a graph function and v a vertex with p^ > 0. Then there exist edges {u, v) and {v, w) 
in Go, such that 

I Pv . Vu ~\~ Uw 

(« 9.+ y<^—. 


Proof Let u, w be vertices such that auv = and a,uw = (i-e., (rt, v) and (n, w) are incoming and 

outgoing edges of maximum weight at v). Note that p^ = — ctuv 

To prove ^ we let u' be a vertex such that the edge («', v) achieves the maximum in (|^, i.e., = x^, 

and write 


(V) 


, Pv R . Pi> 

VV + -^ — VU' + Otu'V — + — 

, R , ^VW ^UV 

— Uu' + O-u'v ~ O-u'v “I- 2 - 

^ R , ^vw T Oiu'v 

< Uu' — Oi^'v “I X ! 


where in the first line we used equation Q and in the last line the fact that a^’v < otuv Now two further 
applications of equation Q give 


OL'i. 


= Vw-yv + a 


R 


Oiu'v - Vv Vu' T (^u'l 


Plugging these into Q yields 

, Pv ^ R , yw-yu'+ Vu' + yw , - Olu'v ^ Vw + Vw 

yv + <yu' - H- 2 -^-2-^-2- “ -2-’ 

where the last inequality follows from the facts that is raising balanced and is maximal among 
incoming edges at u. This completes the proof. □ 


Before proving Proposition]^ we pause to observe a consequence of Lemma 10 

Corollary 11. i/max = 0. 
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this is < < 


Proof. A raising operation applied at some vertex v increases to + ^. By Lemma 
ymax- Therefore ymax can never increase during the raising phase. Choose a vertex u so that y^J = ymax 
initially. Then u can never rise, and so r* = 0. Thus ymax = Vu = —fu = 0- 


We conclude this subsection by proving Proposition!^ 

Proof of Proposition^ Let v be any vertex with raising imbalance pv > 0. By Lemma [T0| we have 

Pv ^ Vu Uw , _ , 

S 2 Uv Si l/max ymin — n- 

Together with the fact that, by definition, /i > 0, this completes the proof. □ 


5.3. Proof of Proposition [9[ The reverse inequality. Proposition]^ is considerably more delicate. It will 
make crucial use of the two-dimensional representation introduced earlier, as well as one additional concept 
which we call “momentum,” whose role will emerge shortly. The momentum of a vertex v is defined by 

(8) my := max{a!„.y - x„}, 

UlU^V 

where Xy is fhe level of v as defined above. 

One key properfy of momenfum is fhaf if can be upper bounded af a vertex in ferms of fhe sum of fhe 
momenfum and fhe local imbalance af fhe predecessors of fhe verfex in fhe graph. (The form of fhis bound 
is complicafed by a dependence on fhe levels of fhe verfices; see pari (i) of Lemma [T^) The second key 
properly of momenfum is lhal, along any edge, heighl can increase only af a rale bounded by momenfum and 
local imbalance. (Again Ihere are additional ferms complicafing fhe dependence; see pari (ii) of Lemma [T^) 
Pulling Ihese Iwo properties fogelher will show lhal, if fhe heighl difference belween some Iwo verfices 
in fhe graph is subslanlial, Ihen fhe aggregate local imbalance py musl be large, which is exaclly fhe 
conlenl of Proposition]^ 

Lemma 12. The momentum satisfies the following inequalities: 

(i) For any vertex v, my < max {O, maxu:u^y^y^<y^ (my + Pu + Xy - x^)}. 

(ii) For any edge {u, v) in Ga, Vy < Vy + my + py + Xy - a^y. 

Proof Bolh parls rely on fhe following elemenlary observafion. For any edge (tt, v), by fhe definition of py 
we have 

( 9 ) ayy < max + Pu = my + Py + Xy, 

u'\u'^u 

where we have used fhe definifion of momenfum (]^. 

For par! (i), we fhus have 

my = max {ayy - Xy} < max {m„ + py + Xy - Xy}. 
u\u—^v u:u^v 

Now note lhal if yy > y^ we have, recalling equalions (]^Q, 

( 10 ) ayy - Xy =yy-yy+ - Xy <0. 

This justifies fhe reslricfion on fhe righl-hand side of (i) fo maximization over u s.f. yy < yy. This completes 
fhe proof of (i). 

For part (ii), apply equations (]^]^ fo deduce 

yv=yu+ Olyy - Oiyy < yu + my + Py + Xy - , 


as required. □ 
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To complete the proof of Proposition we need to translate Lemma [T^ which gives local bounds on the 
increase of momentum and height between adjacent vertices, into global bounds in Ga, as specified in 
Lemma 13 below. The key to this translation will be to apply part (ii) of Lemma 12 to a carefully chosen 
sequence of edges, for each of which the extra term Xu — o-uv the lemma is non-positive and which 
together span the heights in the graph; these edges will be identified in Lemma [T4| below. 


For a vertex v, let Sy denote the set of vertices whose height is strictly smaller than v, i.e., Sy = {u : yu < 
yy}. Also, let p{Sy) = Pn- Finally, define 


( 11 ) 


Zy := max{a:„ : < y„} - Xy. 


This definition has the consequences that Zy > 0 and 

(12) [yy ^ yy^ [Xy T Zu ^ Xy T Zy^. 


We are now ready to state our global bounds. 

Lemma 13. For any vertex v, the following two bounds hold: 

(i) my < p{Sy) + Zy-, 

(ii) yy ?/min ^ ' p(Sy). 


Proof. Both parts (i) and (ii) are proved by induction on vertices v in order of height. We begin with part (i), 
which will be used in the proof of part (ii). 

Part (i). We use induction on the height yy of v. If yy = min„ yy, the statement is true since my < 0 (this 
follows from part (i) of Lemma [T2b and the right-hand side is non-negative. Assuming yy > min^j yy, and 
beginning with part (i) of Lemmia]!^ we may write 

my < max jo, max {rriy + py + Xy — Xy}\ 

'' u:u^v,yu<yv ’ 

< maxjO, max {p{Sy) + Zy + Py + Xy - Xy]] 

u:u^v,yu<yv 

< max jo, max {p{Sy) + Zy + Xy - Xy}], 

u:u—>-v,yu<yv 

where in the second line we applied induction to bound my since yy < yy, and in the third line we used the 
fact that u e Sy \ Sy to deduce that p{Sy) + py < p{Sy). 

To complete the proof we need only apply Eqn. ( [T^ to conclude that for all u in the last maximization, 

Xy Z.y ^ Xy -j- Zy. 


Part (ii). Again we use induction on the height of v. The base case is yy = ymm, in which case the statement 
holds trivially. 

Now let V be any vertex with ymin < yy, and suppose part (ii) is established for all u with yy < yy. Let U 
be the subset of {u : yy < yy} having Xy = xjj, where xu = max{x„ : yy < yy}. Of course U %, and 
Zy = 0 for all u G U. We require the following additional fact: 

Lemma 14. There is a vertex u' G U having an edge [u', v') where v' ^ U, a^/yi > xjj, yy' > yy. 
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We defer the proof of Lemma 14 to the end of the subsection and continue with the proof of Lemma [T3|^ii). 


edge 

(u', v') provided b> 

Lemma 

14 we have 

< 

yu' + ixiu' + Pu' + 

^u'v' 

< 

Vu' + + Pu' 



< 

yu' + p{Su') + Zu' 

+ Pu' 


= 

Pu' + P{Su') + Pu' 



< 

ymin + |<S'n'|p('S'ii') 

+ p{Su' 

+ Pu' 

< 

2 /min T |'S'i;|p(»Si;). 




In the second line here we have used the guarantee of Lemma 14 that - > xu\ 'm the third line we have 


used part (i) of the current lemma; in the fourth line we have used the fact noted earlier that = 0 for any 
u' G [/; in the fifth line we have used the inductive hypothesis applied to (which is strictly less than i/„); 
and in the last line we have used the fact that u' £ Sy\ Su’ ■ This completes the inductive proof of part (ii) 
of the lemma. □ 


Our main goal in this subsection, Proposition]^ now follows trivially from Lemma [T^ 
Proof of Proposition^ By part (ii) of Lemma 13 we have, for any vertex v, — i/min < 
It remains only for us to supply the missing proof of Lemma [T4] 


Proof of Lemma jJ?] The proof will follow from two claims. 

Claim (i): Every vertex u £ U has an outgoing edge (n, w) with = xu- 

To state the second claim, define fhe graph G' formed by fhe sef of edges {{u, w) : u £ U, > xu}- 
Claim (ii): There is an edge (u, w) £ G' such fhaf w ^ U. 

Claim (ii) implies fhe lemma because (u, w) £ G' implies > xu', this last implies also that Xju > xu', 
and then w cannot have < yv as it would then be in U. 

Proof of Claim (i): Since u £ U, we know that yu < 0 and hence some raising must eventually be performed 
at u, i.e., eventually > 0 and then p^ = 0. But once this happens, the same argument as in the proof 
of Proposition 1^ shows that = 0 at all future times, and hence in (at the conclusion of the raising 
phase) we must have Pu = Pu — 0- Thus if (u, w) is a maximum outgoing edge at u in a^, we have 

o^uw = = xu- □ 

Proof of Claim (ii): By Claim (i), every u £ U has an outgoing edge in G'. Suppose for contradiction that 
w £ U for all edges of G'. Then in G' there is a sink sec on vertices U' C U. (U' may be as small as a 
single vertex with a self-loop.) 

Let <5o = - max^j/g[// Note that 6o > 0. Also, let (5i = min{a^^, - > 0}; if this set 

is empty let = oo. Finally, select 0 < <5 < min{<5o, (5i}. 

Now fix any infinife fair sequence of raising operafions, and consider fhe firsf operation in fhe sequence 
thaf increases any y^, u £ U', fo a value larger fhan —6. (This is well defined as all yu,u £ U' increase 
monofonically fo zero.) Lef u' be fhe verfex raised by fhis operafion, and lef a be fhe graph function 
immediafely affer fhe operafion and y fhe heighf function corresponding fo a. 

By consfrucfion fhere is an edge (f, u') in G', wifh t £ U'. Applying equation (j^ fo fhe edge (f,«'), we see 
thaf 

(13) atu' = + yu' -yt>xu + yu' -yt> xu, 

where fhe firsf inequalify is because fhe edge is in G', and fhe second inequalify is by choice of u'. (Nofe 
fhaf equalify can hold here only if f = tt'.) 
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Now let (tt', w) be a maximum (in a) outgoing edge at u' . Since a nontrivial raising operation has just been 
performed at u', it must be the case that w ^ u' . (Raising preserves the order of weights among outgoing 
edges in a. If a self-loop is a maximal outgoing edge from u' in a then it was also maximal before raising. 
But then there could have been no nontrivial raising at u'.) Also, since we have just raised at u', p^, = 0, so 
Oiu'w > Otn' ■ Combining with ( [T3] ) we have 

(14) Uu'w > Xu- 


Applying equation ^ to the edge {u', w) we get 

^u'w ^u'w “h Uu' Uw ^ CX-u'w 4“ Uu' — Oiu'w ^ ^ Xu 6, 


where the first inequality is because heights are nonpositive, the second is by the choice of u', and the third 
is by equation (141. But since (5 < this in fact implies that > xu- Hence the edge (tt', w) belongs 
to G', and since U' is a sink sec it must be the case that w G U' . But this implies that a^tyj < x^ = xu, and 
therefore in fact that = xu', and also, by choice of u' , that < Vu'- Hence 

oiu'w oiy^/yj yu' 4“ Vw Xu yu' 4“ Pw ^ Xu j 


which in light of ( [14] ) gives us the desired contradiction. This completes the proof of Claim (ii) and therefore 
the lemma. □ 


5.4. Proof of Theorem]^ We are now finally in a position fo prove our main resulf. Theorem]^ 


Proof of Theorem^ If suffices fo show fhaf, wifh high probabilify, 0{n^ log{pn/e)) raising operations 
suffice fo achieve an e-raising-balanced graph function. As we observed af fhe beginning of fhe section, by 
symmefry fhe same holds for fhe lowering phase, and hence fhe oufpuf of fhe algorifhm will be e-balanced 
w.h.p. 

To analyze fhe raising phase, we index all quantifies by t, fhe number of raising operafions fhaf have so far 
been performed. Thus in parficular y^ (t) is fhe heighf of verfex v affer t raising operafions. We also infroduce 
fhe potential function 'k(f) = — yv{t)- Note fhaf 'l'(f) > 0 and fhaf 'l'(f) decreases monofonically fo 0 
as f —)• oo. Moreover, by Proposifion[^we have 

T'(f) = > h{t) > p^{t)/2. 


Hence in order fo achieve raising imbalance af most e after t raising steps it suffices fo ensure fhaf 'k (f) < 
e/2. 

Since a raising operafion af verfex v increases by p^ j2, fhe expecfed decrease in T'(f) in one raising 
operafion is 


E[^(f)-^(f + l)] 


1 

2n 


E 


Pvit) > 


h{t) 

2n{n — 1) 


2/min(f) 

2n{n — 1) 




where in fhe firsf inequalify we have used Proposition [^ in fhe nexf step Corollary [TT] and in fhe lasf 
inequalify fhe facf fhaf ymin is less fhan fhe average of fhe y^. Iferafing for t sfeps yields 


E|>t(t)i<(i-A)T(o). 


To gel an upper bound on 41(0), we observe fhaf 


4'(0) = - ^ yvio) < -ra?/min(0) = nh{0) < n{n - 1) ^ /O^(0) < n^p^{0) < n^p, 

V V 
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where we have again used Corollary 11 and Proposition]^ (Recall that p = max{/9^(0),p^(0)} is the 
imbalance of the original matrix.) Hence, for any 6 G (0, 1), after t = 6n^ ln(2pre/e5) raising operations 
we have 


E[^(f)] < eS/2. 

By Markov’s inequality this implies that 'h(f) < e/2 with probability at least 1 — 6. To obtain the result 
w.h.p., it suffices to take 6 = o(l). □ 


6. Open problems 

We close the paper with some remarks and open problems. 

1. The introduction of raising and lowering phases into the Osborne-Parlett-Reinsch algorithm is trivial 
from an implementation perspective (the effect being simply to censor some of the balancing operations), 
but seemingly crucial to our analysis. If balancing operations of both types are interleaved, we no longer 
have a reliable measure of progress. It would be interesting to modify the analysis so as to handle an arbitrary 
sequence of balancing operations, and on the other hand to investigate whether the introduction of phases 
actually improves the rate of convergence (both in theory and in practice). Similarly, it would be interesting 
to investigate the effect of choosing a random sequence of indices at which to perform balancing, rather than 
a deterministic sequence as in the original algorithm. 

2. Recall that, without raising and lowering phases, the balanced matrix to which the algorithm converges 
is not unique (except, by definition, in the UB case). It would be interesting to investigate the geometric 
structure of the set of fixed poinfs, which apparenfly can be rafher complicafed. 

3. As we have seen, our O(n^) bound on fhe worsf case convergence time is essentially opfimal. While fhis 
goes some way towards explaining fhe excellenf performance of fhe algorifhm in pracfice, more work needs 
to be done fo explain ifs empirical dominance over algorifhms wifh faster worsf-case asympfolic running 
times. In particular, can one identify fealures of “fypical” mafrices fhaf lead fo much faster convergence? 

4. We have analyzed only fhe Loo varianf of fhe Osbome-Parleff-Reinsch algorifhm. If would be inferesfing 
to analyze fhe algorifhm for ofher norms Lp, in parficular L 2 and Li. (In fhe latter case, fhe problem corre¬ 
sponds fo finding a circulation—in fhe sense of nefwork flows—fhaf is equivalenf fo fhe inifial assignmenf 
of flows fo fhe edges of fhe graph.) 
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Appendix A. Proof of convergence 

In this section we present an alternative proof of convergence of the original Osbome-Parlett-Reinsch Lqo 
balancing algorithm. As remarked earlier this was first proved by Chen O using a compactness argument. 
In the interests of making the present paper self-contained, we give here an alternative proof that is more 
combinatorial in nature. For this proof only, we explicitly allow more general sequences of balancing 
operations, subject only to the “fairness” constraint that every index i appears infinitely often. (Clearly, the 
random sequence discussed in the main text has this property with probability 1.) Note, however, that this 
proof does not apply to the two-phase version of the algorithm which we analyze in this paper because the 
sequence of balancing operations in that version is not fair. We prove convergence of the two-phase version 
in Section m 

Theorem 15. Consider the Osborne-Parlett-Reinsch L^o balancing algorithm applied to any irreducible 
(real or complex) input matrix A, and assume that balancing operations are performed infinitely often at 
all indices i. Then the algorithm converges, i.e., if is the sequence of matrices after each step of the 
algorithm, then there exists a balanced matrix B equivalent to A such that -A B as t ^ oo. 

Proof. We work in the framework of Section representing A as a graph Ga with an associated graph 
function a, where auv = log \ auv\ (and edges (rt, v) with Ouv = 0 are omitted). We let a™ and a™' denote 
the maximum incoming and outgoing edge weights at vertex v in G^, we also set = max{ajj*, 

The key ingredient in the proof is the following partial order on graph functions defined on a common 
sfrongly-connecfed graph G. For fwo such graph funcfions a, 7 , we say fhaf a < 7 if, for fhe largesf 
weighf w such fhaf / G^, we have G'f C G^. 

Now, if a decreasing sequence al*! of graph funcfions in fhis order has fhe properfy fhaf all values Oul 
in fhe funcfions are bounded below by a common value b, if is clear fhaf fhe sequence has a limif. The 
convergence of fhe iferafive balancing algorifhm will fherefore follow from fhe nexf fwo lemmas. Call a 
balancing operation af i non-trivial if a™ / 0 :°“', so fhaf fhe operafion acfually changes fhe weighfs. By 
definifion a is balanced iff no non-frivial balancing operafion is available. 
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Lemma 16. Non-trivial balancing operations are strictly decreasing in the above order on graph functions. 


Proof. Let a be the eurrent graph funetion, and let n be a vertex at whieh a non-trivial balaneing operation is 
performed. Let a denote the new graph funetion after the operation. We have a™™ = — \cdf — q;°“‘|/2, 

and The balaneing operation affeets only edges ineident at v. Before the balaneing, at least 

^max 

one of those edges belongs to Ga , while none belong to for any w > After the balaneing, none 

Q,max 

of those edges belong to Ga ■ Thus the largest value of w for whieh G'f and differ is w = and 


G 


Ct 

a 


max 

V 


cGo 


□ 


Lemma 17. Given any graph function a, there is ab > —oo such that no sequence of balancing operations 
can create an edge of weight less than b. 


Proof. A balaneing operation at vertex v reduees Q!™“, so no sueh operation ean inerease the maximum 
edge value above its original value in Ga, say mi. Let wi be the least average weight of any eyele in Gq,; 
note that wi remains invariant under any sequenee of balaneing operations. Consider the value of any edge 
{u, v) under any graph funetion a' that is reaehed by balaneing operations from a. Let {u, v) belong to 
some (simple) eyele of average weight W 2 and length i. Then + (£ — l)mi)/i > W 2 > wi, so 
ot'uv ^ wi — {£ — l)(mi — wi) > wi — {n — l)(mi — wi). This eompletes the proof of the lemma and the 
theorem. □ 



